Magnification of small objects using a microscope is well known. Microscopes facilitate magnification of small objects to thereby allow details of the small objects to be rendered visible. At any given magnification, a microscope has a corresponding field of view. In general, the greater the amount of magnification the smaller the corresponding field of view relative to the object. Similarly, and as represented in FIG. 1, at any given focal distance, a microscope objective lens (10) has a corresponding focal plane with a depth of field (11) (that is, a Z-axis range within which objects will appear to be in focus). In general also, the greater amount of magnification the smaller the corresponding depth of field relative to the object. The capture of single digital images of these microscope fields of view is well known, and the art is experienced with the capture and display of stacks of images at a single object position to record depth of field image content. Such images are used for example in confocal microscopy instruments to image through objects by varying the Z-axis focal position of each image in the image stack at a single X, Y planar location.
In the early microscope technology, around 1750, microscope specimens were placed between 2 small, thin circular glass plates, and mounted on long ivory “sliders” that could be pulled back and forth in a slot under the microscope objective lens. With today's technology the sliders have been replaced by rectangular glass “slides” as a mounting structure, the object specimen is placed on the slide and sometimes covered by a thinner glass “coverslip”. These glass slide mounting structures are not flat over their entire surface area, i.e within the tolerances of the depth of field of a common 40× to 100× microscope objective lens. They are thicker in some portions than in others and sometimes have a warp or curvature. This creates a significant problem in the construction of a virtual microscope slide in contrast to taking a single field of view image. This is because in most instances the Z-axis focal plane of the objective will not be positioned in the same cross sectioned portion of the specimen, and thus not be “in focus” across the entire surface of the slide, i.e in adjacent planar X, Y field of views, without adjusting the specimen in the Z-axis dimension in some manner. For example, in the simple case of one end of the slide being thicker than the other end, all other factors being equal, and assuming the stage support is flat, this produces a slope across the slide with regard to positioning the same portion of the cross section of the specimen in the objectives focal plane. This is not a problem for single field of view multiple Z dimension images because the slope is not apparent in the small field of view. Another aspect of this problem relates to the stage support. Stages commercially available are often not parallel and flat across the complete working distance of the commonly used glass microscope slides. Also microtome sectioning does not produce uniformly thick sections. So in cross section the thickness of the specimen object varies. Thus the proper focal plane can vary from place to place on the slide from a multitude of factors. The focal distance position is determined by the microscope objective lens, and although the lens could move to adjust the focal plane position, it is common to move the stage platform that holds the glass slide structure up and down in the Z direction to obtain the optimal focal plane for a given specimen location and single field of view, or image tile. Thus, as is well known in the art, the focal plane position in the Z-axis, relative to the microscope slide planar surface and deposited specimen thereon varies substantially from point to point for accurate focus in a give specimen.
Virtual microscope slides are also known. U.S. Pat. No. 6,272,235 B1 (entitled Method and Apparatus for creating a virtual microscope slide), the contents of which are incorporated herein by this reference, teaches the creation, storage and Internet or intranet display of virtual microscope slides. As taught therein, a virtual microscope slide typically comprises a digitized magnified view of part or all of a microscope slide and an object (such as a biological specimen) disposed thereon. Virtual microscope slides; when created, overcome limitations of the microscope optical field of view restrictions; they have a data structure for storing the digital images from different parts of the slide to enable the reconstruction of an X, Y planar view from composite image parts; and when viewed, overcome the limitations of the finite size of computer terminal display screens, with Internet or intranet viewer software that seamlessly and rapidly allows the user to navigate from place to place in the virtual image, and to zoom the virtual image to mimic changing of magnification with different microscope objectives. Prior art virtual slides allow computer viewing to mimic the viewing and inspection process obtained by looking through a real microscope with respect to viewing abutted, aligned X, Y planar image views.
As taught in the aforesaid patent, the area of the object digitized is comprised of multiple, adjacent, microscope objective optical fields of view captured at a single Z-plane focal distance. In some cases thousands of microscope objective optical fields of view are recorded to represent the virtual microscope slide. As taught in the aforesaid patent, the individual digitized fields of view are referred to as tiles. The chosen Z-plane object position varies for a given tile with the X, Y location on the microscope slide, and as taught in the aforesaid patent, is obtained as a representative, reasonably optimum, focal position choice by an automatic focusing determination on individual image fields, or by extension from previously determined focal positions of nearby image fields. The object is digitized and the resulting images stored in a data structure that allows for subsequent retrieval for review or image processing.
Because of the limitations of the microscope objective lens optics field of view, the capture event of virtual microscope slide tiles is always restricted to only a small part of the object in at least one planar dimension. As further taught in the aforesaid patent, the digital capture was with a 3 color chip CCD sensor, which enabled the same object area sampled pixel point in and individual tile to be captured as 3 identical color pixels, in register with each other. In an alternative embodiment of a scanning method not taught in the aforesaid patent, a line sensor, e.g with dimensions of 3×2098 pixels, could be used and moved in one direction at a constant speed, and the sampling could be performed to acquire a series of tiles of dimension 3×2098 stored in computer memory to form a larger image segment. However, this image segment is still limited in one direction by the optical field of view, and subsequent adjacent tiled image segments are acquired to construct the virtual microscope slide. In this case the 3 pixels at each given position along the line sensor provide different color sensing, thus there is a small loss of color and spatial information with this method. As is known in the art, other combinations of sensor sampling can be obtained. However to construct a truly virtual microscope slide image capture that can be reconstructed to abut captured image portions, the method must overcome the limitation of the very small optical field of view in at least one dimension of the object plane of the microscope objective lens at high magnifications. Typically this is accomplished by either moving the stage or the microscope objective to cover the object area and construct the digitized image data structure.
It may be further appreciated that the digitized image data structure may be stored in numerous ways to facilitate future viewing. One method may be to simply store each capture event in a very large contiguous digital memory or storage. In this case the subsequent viewing may be accomplished by simply indexing this memory and displaying standard 2 dimensional images, e.g of X by Y pixel size, on a computer screen. However, with this method the virtual slide Internet server memory requirements become very large. As described in the aforesaid patent a tiled data structure is more efficient of server memory and Internet transmission speed.
It is additionally taught in the aforesaid patent, that the standard computer video display will also only display a small portion of the total virtual slide at the original capture resolution, or highest magnification. To overcome this, various methods of image data structure and storage have been described, and typically the viewer program can zoom in and out to display high and low magnification fields, and can cache portions of the virtual slide image data that have been previously transmitted from digital storage or an Internet server and viewed. The viewer display programs must handle the indexing and addressing to bring in only the user requested image portions. Also, the virtual microscope slide can be scrolled in various directions and thereby mimic movement of the object/slide with respect to the microscope objective lens. Such virtual microscope slides can be used for a variety of purposes, including education, training, and quantitative and qualitative analysis.
For many applications, such virtual microscope slides work well, and especially with specimens that are of relatively uniform thickness and with features of interest that tend to be within a single depth of field. Such virtual microscope slides solve the first of two significant technological issues of virtual microscopy; the first being the issue of aligning small adjacent image segment views and displaying them seamlessly in X, Y registration. For any given level of magnification, the microscope can be automatically focused on such a specimen and the corresponding single focal plane image digitally captured and stored for later retrieval and use.
When the specimen exhibits significantly varying depth, however, and/or where features of interest are more widely spaced with respect to depth, prior art virtual microscope slides may contain images that are not fully focused with respect to one or more desired elements. This is the second major technical issue with virtual microscopy; the issue being finding the proper focal plane to represent the image in the first place, or alternatively including the Z-axis dimension across the entire slide and in so doing in either case, overcoming the problem of a non-flat microscope glass slide support and the problem of tissue sectioning and deposition irregularities that change the position of the optimum focal plane relative to the planar X, Y surface of the glass slide. Consistent with the inherent problems of this second issue, obtaining stacks of Z-plane images in an uncoordinated fashion from many different non-abutting object positions, without an integrated virtual slide data structure is both difficult to adequately store and retrieve, and to view in a coherent fashion in an Internet or internet environment. For example, and with reference to FIG. 2, a microscope slide (21) can bear a specimen having portions (22) of relatively even depth, or Z-axis position, and/or portions (23) that vary significantly with respect to depth. While some portions (22) may reside within the depth of field (11), other portions (26 and 27) that extend above or below the depth of field (11) will likely appear unfocused in the resultant image. Similarly, features of interest (24) that occur within the depth of field (11) will appear focused but features of interest (25) that are outside the depth of field range may appear unfocused. Regardless of whether such a virtual microscope slide is being used academically, for tissue microarray imaging, as in patent U.S. Pat. No. 6,466,690 B2, or with diagnostic intent, unfocused elements often render such an image unsuitable for the desired activity.